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1  Objectives 

This  research  project  studied  problems  of  object-based  image  reconstruction  and  feature 
enhancement.  Identification  and  extraction  of  object  features  is  a  critical  aspect  of  automatic 
target  recognition  (ATR)  systems.  The  robust  estimation  of  geometric  features  becomes 
increasingly  difficult  as  data  quality  and  quantity  are  reduced.  As  such,  it  is  important 
to  develop  methods  which  take  into  account  from  the  outset  the  feature-oriented  nature  of 
scenes  during  the  inversion,  image  formation,  and  enhancement  process.  A  limitation  to  the 
practical  use  of  such  methods  is  the  their  computation  cost,  thus  efficient  algorithms  are  also 
important. 

2  Status  and  Accomplishments 

The  incorporation  of  object  geometry  or  shape  during  image  formation  and  enhancement  is 
a  challenging  but  an  important  problem.  To  address  these  challenges,  we  developed  methods 
which  directly  model  and  incorporate  object  shape  in  a  scene.  In  particular,  a  level  set  model 
was  used  to  capture  object  geometry  and  was  coupled  with  an  appropriate  pixel  model  which 
captured  scene  texture  or  intensity.  Thus,  an  explicit  treatment  of  object  shape  was  in  our 
formulation. 

Part  of  this  work  developed  and  applied  such  models  for  statistically-based  segmentation 
tasks.  For  example,  we  are  studied  the  problem  of  segmenting  laser  radar  range  imagery.  To 
accomplish  this  task  we  combined  a  contour-based  object  model  with  a  physically-motivated, 
statistical  intensity  model  to  obtain  maximum  likelihood  estimates  of  the  scene.  We  incor¬ 
porated  prior  knowledge  of  both  background  and  target  behavior  to  directly  estimate  both 
target  boundary  as  well  as  the  scene  range  profile.  In  contrast  to  simple  smoothing  of  the 
image,  our  approach  directly  used  a  statistical  range  anomaly  model  in  an  EM  algorithm  to 
estimate  and  correct  for  pixels  with  anomalous  range  values  -  a  bane  of  laser  radar  imagery 
-  as  well  as  directly  estimating  object  boundary  location. 

A  critical  challenge  in  the  application  of  such  contour-based  methods  to  practical  prob¬ 
lems  is  their  computational  cost.  As  such,  we  have  developed  new,  extremely  fast  imple¬ 
mentations  of  the  level-set  curve  evolution  method.  These  methods  appears  to  be  the  first 
implementation  allowing  real-time  use  of  the  level  set  method  for  image  processing  tasks. 
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We  have  developed  a  Matlab-based  video  segmentation  demonstration  to  illustrate  the  true 
real  time  ability  of  the  new  approach.  This  work  won  the  best  paper  award  at  ICASSP  in 
2005. 

A  related  aspect  of  this  work  was  the  development  of  object  or  shape-based  prior  models. 
These  models  allow  the  incorporation  of  target  and  scene  specific  information  concerning 
object  boundary  behavior  into  processing  problems.  In  typical  curve  evolution  methods 
stabilization  of  the  boundary  to  noise  is  accomplished  through  inclusion  of  a  prior  term 
which  simply  penalizes  overall  boundary  length.  Such  a  generic  prior  may  be  inappropriate 
in  some  applications,  but  does  not  capture  scene  specific  object  information  that  may  be 
available.  We  developed  a  feature-histogram-based  shape  prior  for  use  in  curve  evolution- 
based  segmentation  methods.  Such  feature-histogram  representations  of  object  shape  have 
had  great  success  in  classification  tasks,  and  our  work  represents  their  first  application  to 
continuous-valued  inferencing  problems,  such  as  segmentation.  We  continued  to  extend 
our  approach  to  allow  the  development  of  prior  shape  models  capturing  the  interaction  of 
multiple  scene  objects.  In  addition,  we  also  developed  methods  for  the  direct  incorporation 
of  topographic  object  constraints  into  the  segmentation  process. 

A  final  aspect  of  our  work  focused  on  shape-based  dynamic  inverse  problems,  where  the 
scene  under  consideration  evolves  in  time.  We  have  used  the  problem  of  real-valued  dy¬ 
namic  tomography  as  an  paradigm  of  such  problems.  The  challenge  of  creating  imagery 
from  projection  type  data  arises,  for  example,  in  Synthetic  Aperture  Radar  (SAR)  image 
formation.  In  our  approach  we  have  directly  modeled  the  evolution  of  object  shape  in  time 
and  included  this  dynamic  model  in  the  image  formation  process.  We  have  developed  dy¬ 
namic  models  reflecting  specific  classes  of  object  motion  (i.e.  smoothness)  and  incorporated 
these  into  inversion  tasks.  In  tomographic  problems,  a  complete  set  of  projection  data  at 
each  time  instant  is  typically  required  by  standard  inversion  techniques  to  produce  artifact 
free  images  and  reliable  quantitation.  This  requirement  is  almost  never  met  in  challenging 
dynamic  problems.  For  these  extremely  difficult  problems  we  have  had  good  success  in  pro¬ 
ducing  excellent  reconstructions  using  our  approach.  In  particular,  we  have  demonstrated 
good  target  boundary  localization  as  well  as  robust  intensity  estimation  with  as  little  as  one 
projection  per  time  instant. 
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Transform,”  IEEE  Trans.  Image  Processing  Yol.  15,  No.  3,  pg.  582-591,  March,  2006. 
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2.  Y.  Shi  and  W.  C.  Karl,  “A  Real-Time  Algorithm  For  The  Approximation  Of  Level- 
Set-Based  Curve  Evolution,”,  IEEE  Trans.  Image  Processing.,  in  press. 

3.  Y.  Shi  and  W.  C.  Karl,  “Topological  priors  for  shape  based  segmentation,”  in  prepa¬ 
ration. 

4.  H.  Feng,  D.  A.  Castanon,  W.  C.  Karl,  “Ladar  Range  Image  Processing  using  Curve 
Evolution  and  Maximum  Likelihood  Estimation,”  IEEE  Trans.  Image  Processing,  in 
press. 

5.  R.  Weisenseel  and  W.  C.  Karl,  R.  C.  Chan,  and  T.  J.  Brady,  “A  Variational  Approach 
to  Multi-modality  Subsurface  Data  Inversion  and  Fusion  Based  on  Shared  Image  Struc¬ 
ture,”  Subsurface  Sensing  Technologies  and  Applications,  Vol.4,  No.4,  pgs  373-392, 
October  2003. 

4.2  Reviewed  Conference  Proceedings 

1.  A.  Litvin,  W.  C.  Karl,  J.  Shah,  “Shape  and  appearance  modeling  with  feature  distribu¬ 
tions  for  image  segmentation,”  Proceedings  of  the  2006  IEEE  International  Symposium 
on  Biomedical  Imaging,  Arlington,  VA,  April  6-9,  2006. 

2.  H.  Feng,  W.  C.  Karl,  D.  A.  Castanon,  “LADAR  Range  Image  Segmentation  using 
Curve  Evolution  and  Maximum  Likelihood  Estimation,”  Proc.  of  the  Thirty-Ninth 
Asilomar  Conference  on  Signals,  Systems,  and  Computers,  October  30-November  2, 
2005. 

3.  Y.  Shi  and  W.  C.  Karl,  “Real-time  tracking  using  level  sets”,  IEEE  Computer  Society 
Conference  on  Computer  Vision  and  Pattern  Recognition,  2005  (CVPR  2005),  Volume 
2,  Pgs.  34-41,  June  20-25,  2005. 

4.  A.  Litvin  and  W.  C.  Karl,  “Coupled  shape  distribution-based  segmentation  of  multi¬ 
ple  objects,”  Proceeding  of  Information  Processing  in  Medical  Imaging  (IPMI  2005), 
Glenwood  Springs,  Colorado,  2005. 

5.  Y.  Shi  and  W.  C.  Karl,  “A  fast  level  set  method  without  solving  PDEs,”  Proc.  2005 
IEEE  Int’l  Conf.  on  Acoustics,  Speech,  and  Signal  Processing,  pg.  II-97-II-100, 
Philadelphia,  March  20-23,  2005.  Student  Paper  Award  Winner. 

6.  Y.  Shi  and  W.  C.  Karl,  “Differentiable  Minimin  Shape  Distance  for  Maintaining  Topol¬ 
ogy  in  Curve  Evolution,”  Proc.  2004  SIAM  Conference  on  Imaging  Science,  Salt  Lake 
City,  Utah,  May  3-5,  2004. 

7.  A.  Litvin  and  W.  C.  Karl,  “Using  Shape  Distributions  as  Priors  in  a  Curve  Evolution 
Framework,”  Proc.  2004  IEEE  Int’l  Conf.  on  Acoustics,  Speech,  and  Signal  Processing, 
Montreal,  pg.  III-25-III-28,  May  17-21,  2004. 
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8.  Y.  Shi  and  W.  C.  Karl,  “Shape  reconstruction  for  unorganized  points  with  a  data- 
driven  level  set  methods,”  Proc.  2004  IEEE  Int’l  Conf.  on  Acoustics,  Speech,  and 
Signal  Processing,  pg.  III-13-III-16,  Montreal,  May  17-21,  2004. 

9.  Y.  Shi  and  W.  C.  Karl,  “Tomographic  Reconstruction  of  Multiple  Dynamic  Objects,” 
Proc.  2004  SIAM  Conference  on  Imaging  Science,  Salt  Lake  City,  Utah,  May  3-5,  2004. 

10.  Y.  Shi  and  W.  C.  Karl,  “Shape  reconstruction  for  unorganized  points  with  a  data- 
driven  level  set  methods,”  Proc.  2004  IEEE  Int’l  Conf.  on  Acoustics,  Speech,  and 
Signal  Processing,  pg.  III-13-III-16,  Montreal,  May  17-21,  2004. 

11.  Y.  Shi  and  W.  C.  Karl,  “Level-set  methods  for  dynamic  tomography,”  Proc.  2004 
IEEE  Inti.  Symposium  on  Biomedical  Imaging,  pg.  620-623,  Arlington,  VA,  15-18 
April,  2004. 

12.  A.  Litvin  and  W.  C.  Karl,  “Shape  distributions  as  priors  for  image  segmentation,”  in 
Computational  Imaging,  C.  A.  Bouman  and  R.  L.  Stevenson  editors,  Proc.  SPIE,  Vol. 
5016,  SPIE,  Santa  Clara,  CA,  January  20-24,  2004. 

13.  Y.  Shi  and  W.  C.  Karl,  “Multiple  Motion  and  Occlusion  Segmentation  with  A  Multi¬ 
phase  Level  Set  Method,”  in  Computational  Imaging,  C.  A.  Bouman  and  R.  L.  Steven¬ 
son  editors,  Proc.  SPIE,  Vol.  5016,  SPIE,  Santa  Clara,  CA,  January  20-24,  2004. 

14.  Y.  Shi  and  W.  C.  Karl,  “A  Multiphase  Level  Set  method  for  tomographic  reconstruction 
of  dynamic  objects,”  Proc.  2003  IEEE  Workshop  on  Statistical  Signal  Processing,  pg. 
169-172,  St.  Louis,  Sept.  28-Oct.  1,  2003 

15.  A.  Litvin  and  W.  C.  Karl,  “Level  set-based  segmentation  using  feature  histogram  shape 
priors,”  Proc.  2003  IEEE  Workshop  on  Statistical  Signal  Processing,  pg.  153-156,  St. 
Louis,  Sept.  28-Oct.  1,  2003 

5  Inter  act  ions /Transit  ions 

5.1  Conference  Presentations 

•  Presentation  at  the  2005  IEEE  Conference  on  Computer  Vision  and  Pattern  Recogni¬ 
tion,  June,  2005. 

•  Presentation  at  the  2005  Conference  on  Information  Processing  in  Medical  Imaging, 
Glenwood  Springs,  Colorado,  2005. 

•  Presentation  at  the  2005  IEEE  Int’l  Conf.  on  Acoustics,  Speech,  and  Signal  Processing, 
Philadelphia,  2005. 

•  Presentation  at  the  IEEE  Int’l  Conf.  on  Acoustics,  Speech,  and  Signal  Processing, 
Montreal,  May,  2004. 
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•  Presentation  at  the  2004  SIAM  Conference  on  Imaging  Science,  Salt  Lake  City,  May  , 
2004 

•  Presentation  at  the  SPIE  Conference  on  Computational  Imaging,  Santa  Clara,  January, 
2004 

•  Invited  presentation  at  IEEE  Workshop  on  Statistical  Signal  Processing,  St.  Louis, 
Sept,  2003. 

5.2  Transitions 

•  Gave  invited  presentation  at  Workshop  on  ATR  Theory  organized  by  Dr.  Greg  Arnold, 
AFRL,  Acting  ATR  Team  Leader,  AFRL/SNAT. 

•  Interaction  with  Capt.  Virgil  E.  Zetterlind  on  our  fast  level-set  algorithm. 

•  Have  had  discussions  on  transition  of  our  fast  level-set  algorithm  into  the  ITK  toolkit. 

•  Interacted  with  Guna  Seetharaman  and  TJ  Klausutis  on  a  possible  application  of  our 
fast  level-set  method  to  micro-UAV  video  stabilization. 

6  Honors 

Our  2005  ICASSP  Paper  on  our  fast  level-set  method  won  best  student  paper  award. 
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